docs(integrations): INT-1 discovery gate records and capability-matrix checker - #14906
docs(integrations): INT-1 discovery gate records and capability-matrix checker#14906erichare wants to merge 16 commits into
Conversation
…checker Adds design/dedicated-integrations/ for the Dedicated Integrations (1.13) discovery gate (LE-2398, INT-1): a README with the exit-criteria and sign-off tables, a JSON Schema for provider capability matrices, seed matrices for Google Workspace, Microsoft 365/Teams, and Slack with every wave-1 candidate at decision=defer and confidence=low, decision-record stubs for the three substrate decisions and the Google restricted-scope (CASA) decision, and the deferred trigger/webhook track. Adds scripts/ci/check_capability_matrices.py and its pytest in the style of check_execution_principal_matrix.py. The checker turns the gate's exit criteria into CI failures: at most 8 included actions per provider, every scope classified and sourced, every restricted scope decided in an existing record, included actions on a chosen substrate, no high-confidence rows on a non-GA MCP substrate, and low-confidence rows must list open questions. --require-accepted is the gate-close mode. The CI Scripts Tests workflow now also runs on changes under design/dedicated-integrations/. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
WalkthroughChangesDedicated Integrations gate
Estimated code review effort: 4 (Complex) | ~60 minutes Merge Risk: 🟡 Moderate · up to The PR adds discovery-gate contracts and accepted integration decisions, but the current documents still contain credential authorization and collision risks, a Desktop authentication mismatch for Slack actions, and incomplete owner sign-off tracking. These issues can misdirect implementation or permit unsafe credential behavior, so the PR is not merge-ready until corrected or explicitly accepted. Sequence Diagram(s)sequenceDiagram
participant PullRequest
participant CIWorkflow
participant MatrixChecker
participant DecisionRecords
PullRequest->>CIWorkflow: change Dedicated Integrations path
CIWorkflow->>MatrixChecker: run capability matrix checks
MatrixChecker->>DecisionRecords: inspect referenced decision status
DecisionRecords-->>MatrixChecker: return decision metadata
MatrixChecker-->>CIWorkflow: return validation status
🚥 Pre-merge checks | ✅ 6 | ❌ 3❌ Failed checks (3 warnings)
✅ Passed checks (6 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 13.16% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 2 files. (17 skipped: 17 unsupported.) Full details: Test Coverage For New ImplementationsExplanation The PR adds corresponding tests for the new checker. Full details: Test Quality And CoverageExplanation The new tests use pytest and contain meaningful positive and negative checks, but they do not comprehensively cover the new checker. The implementation adds validation for top-level and action required fields, enums, dates, source records, verification programs, decision records, deployment contexts, duplicate and unexpected matrices, sourced action blocks, and CLI exit behavior. The 16 tests cover only selected cap, scope, confidence, future-date, file-error, provider-missing, and draft-record cases. They do not exercise most of those validation paths. Async, frontend, and API-endpoint checks are not applicable to this PR. Resolution Add pytest cases for each checker validation family, including malformed or incomplete top-level and action objects, invalid enums and dates, invalid source and verification-program records, missing or malformed substrate and restricted-scope decisions, invalid action IDs/components/contexts, missing included-action fields, unsourced claim blocks, unknown verification dependencies, duplicate and unexpected matrices, and both successful and failing Full details: Test File Naming And StructureExplanation The pull request adds or modifies frontend test files that do not use Playwright. The seven Resolution Port the changed frontend tests to Playwright tests, use Full details: Excessive Mock Usage WarningExplanation No excessive mock usage is present. The pull request adds one test file,
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Phase 1 of the INT-1 discovery gate. Every Gmail, Drive, and Calendar row now carries its API method schema, scope classification from Google's per-API scope pages, consent and Workspace admin-control behavior, reach, refresh and revocation rules, SDK substrate with the Workspace MCP Developer Preview status, service-specific quota figures, and the verification programs it depends on (brand verification, sensitive-scope verification, CASA with its several-week lead time and 12-month recertification, and the Workspace Developer Preview terms that forbid pre-GA production use). All 22 sources were read on 2026-09-01. Rows stay at decision=defer: the include set is fixed in Phase 4 once the restricted-scope (CASA-or-avoid) decision is taken. Two facts that shape that decision are now recorded: gmail.metadata rejects the q parameter, so there is no narrower Gmail search scope, and the Workspace Gmail MCP server requests gmail.compose (restricted), so MCP cannot avoid CASA for send. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…atrix row Phase 2 of the INT-1 discovery gate. Microsoft: every delegated Graph permission wave 1 needs (Mail.Send, Mail.ReadBasic, Mail.Read, Calendars.Read, Calendars.ReadWrite, ChatMessage.Send, ChannelMessage.Send, Files.Read, Files.Read.All, Sites.Read.All, offline_access) carries admin consent "No" in the permissions reference, so under the gate's mapping they are all non-sensitive; the application-permission variants need admin consent and stay excluded. Rows record the Graph method contracts, Outlook and Teams throttling (1 request per second per chat or channel), the SharePoint resource-unit model, 90-day rotating refresh tokens, the localhost redirect rules for Desktop, publisher verification as the hosted-app dependency, and the Work IQ MCP status (preview, not for production, Copilot license). Slack: user-identity rows target the official MCP server, whose docs admit only directory-published or internal apps and carry no availability label (recorded as an open question), with the Web API as fallback; bot rows run on the Web API because the MCP server issues user tokens only and desktop redirects may not request bot scopes. Rows record the May 2025 non-Marketplace rate reduction on conversations.replies, token rotation semantics, auth.revoke, and the workspace app-approval controls. All rows stay at decision=defer until Phases 3 and 4 fix the include set. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…isions Phases 3 and 4 of the INT-1 discovery gate, drafted at Status: proposed for the release owner to confirm. Substrate: Google runs on the Google APIs SDK because the Workspace MCP servers are Developer Preview whose program terms forbid production use and whose Gmail server requests gmail.compose (restricted); Microsoft runs on Graph REST because Work IQ MCP is preview, not for production, and needs a Copilot license; Slack is proposed mixed (MCP for user-identity actions in pinned mode, Web API for bot actions) on the condition that a Slack source for the MCP server's GA status and its tool names is cited before acceptance, otherwise Web API throughout and INT-9 defers to 1.14. Restricted scopes: the Langflow-owned hosted Google app requests no restricted scope in 1.13. Gmail search is excluded from wave 1 (gmail.metadata rejects the q parameter, so there is no narrower search scope), Drive list and fetch ship on drive.file, and CASA leaves the estimate. A self-managed restricted-scope profile is recorded as the first 1.14 candidate. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…surface list Phases 5 and 7 of the INT-1 discovery gate. connection-contract.md is the INT-2 design for lfx, langflow-base, and Enterprise sign-off: a string connection handle <provider>/<name> on a new channel (not load_from_db) with an env-key encoding that rides the existing headless flat-map contract unchanged; a new ConnectionRefInput and FieldTypes.CONNECTION_REF rather than extending Composio's ConnectionInput; a new CONNECTION_RESOLVER_SERVICE type with a fail-closed lfx.toml override for Enterprise; an ExecutionPrincipal stamped on the Graph with the per-family allow/deny table encoded as a new dimension of execution_principal_matrix.json; a CredentialLease so components never hold refresh tokens; a kebab-case IntegrationError contract; capability metadata shared with the INT-3 manifest field; telemetry without identifiers; Bundle API impact; the INT-2 test plan; and the open questions per owner. frontend-surfaces.md lists 14 existing surfaces to extend and 8 net-new surfaces with owning tickets, the two constraints the design must record (new-tab-plus-polling cannot survive a consent screen; connections must not land in the global-variables table), and the MVP/defer split. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…sue the estimate Phases 6 and 8 of the INT-1 discovery gate, at Status: proposed and draft. KB OAuth connectors: defer adoption of the connection contract to 1.14 and keep KBConnectorSource able to take a connection handle later; ingestion is a background job that needs the non-interactive opt-in from INT-6, and Drive ingestion beyond app-scoped files needs a restricted scope the hosted app avoids. Palette naming: native components use "Product: Verb Object" display names and product-prefixed class names; Composio's bare "Gmail", "Outlook", "Slack", "Slackbot" stay unchanged under the Composio group; the Gmail sidebar group folds into Google, and new Microsoft 365 and Slack groups are added. Estimate: 51 engineer-weeks for INT-1..INT-14 under the proposed decisions (47 if Slack runs on the Web API throughout), inside the plan's 45 to 55 range, with hosted verification lead times recorded as calendar risk and CASA removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Decisions confirmed on 2026-09-01: Google runs on the SDK, Microsoft on Graph REST, the Langflow-owned hosted Google app requests no restricted scope, knowledge-base OAuth connectors adopt the connection contract in 1.13 (overriding the gate's defer recommendation), and native components use "Product: Verb Object" names with new Microsoft 365 and Slack sidebar groups. Slack stays mixed and conditional: the record is accepted on the condition that a Slack source for the MCP server's GA status and its tool names is cited before gate close, so its rows stay deferred for now. Matrices: Google includes Gmail send, Drive list and fetch on drive.file only, Calendar list and create (5 of 8); Gmail search is excluded and the three restricted-scope decisions flip to avoid. Microsoft includes all eight Graph actions. Estimate re-issued at 52.5 engineer-weeks (48.5 if Slack goes Web API throughout); the KB adoption adds 1.5 across INT-10 and INT-11. The restricted-scope test fixture now defers the row it strips the decision from, since the rule applies to included and deferred rows only. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review fixes from the CodeRabbit pass on #14906: - connection-contract: make ConnectionRef.env_key() injective (lowercase snake_case names, loader-time provider collision check); state the headless trust boundary for request-scoped LF_CONNECTION__* keys and add the LFX_CONNECTIONS_ENV_ONLY question; define the expires_at=None path for CredentialLease; align OAuthProfile.kind with the matrix schema's auth_mode enum; add the matching section 11 tests. - substrate-slack: cite the MCP server's PKCE-for-desktop callout and fixed-app-id/no-DCR rules; Desktop slack.user.* use a customer-owned PKCE-enabled app with a Web API fallback verified by INT-9's capture. - substrate-google: scope the record to the five included actions. - matrices: google.drive.fetch MCP note now matches drive.file; microsoft.files.fetch gains Sites.Read.All for its site_id input; slack-mcp-server source note records the auth statements. - frontend-surfaces: A7/A8 mirror the accepted palette decision. - estimate: heading and original total match the table (INT-1, 49). - README: uv run in the gate-close command, text fence tag, and a sign-off table with one row per declared role listing every record; records gain the sign-off rows their Owners lines declare. - checker: validate_sign_offs() turns that drift into a CI failure (README row per role, record listed under each role, record table row per role), with tests.
1bafba1 to
1f06c9e
Compare
…date matrices against the schema Second review round on #14906: - Scope alternatives were modeled as simultaneously required. The schema's scope entry gains role (required | optional | alternative) and condition; every scope in the three matrices is tagged, and the connection contract states how required_scopes and optional_scopes are lifted from the roles. The open Microsoft permission questions on included actions are decided: Mail.Read, Calendars.Read, and Files.Read are the wave-1 profiles, with the wider Files and Sites scopes optional on drive_id/site_id and Mail.ReadBasic recorded as the unused alternative. Slack's search:read is the Web API alternative; users:read, groups:read, and canvases:read are optional on their inputs. The checker requires a role on every included scope, a condition on optional and alternative scopes, no condition on required ones, and at least one required scope per included action. - --require-accepted now walks every record under decisions/ (TEMPLATE.md aside) instead of only the ones a matrix references; validate_sign_offs reports a record that declares non-release owners but has no sign-off table (frontend-surfaces.md and triggers-deferred.md gain theirs), and a missing README is a single error on the CLI path instead of a silent skip. - Every matrix is validated against schema/capability_matrix.schema.json with jsonschema (Draft 2020-12) before the gate rules; a missing validator is an error. The CI Scripts Tests workflow installs it. - Tests for each rule: untagged included scope, no required scope, conditional scope without a condition, required scope with one, empty outputs and an unknown action field under the schema, a draft non-matrix record under gate close, a record without a sign-off table, and a deleted README.
|
Second review round, applied in 32bee29 (32bee29). All three findings were reproduced before fixing. Scope alternatives modeled as simultaneously required. Agreed. The schema's scope entry gains Gate-close validation skipped record requirements. Agreed on all three. Validate against the published JSON Schema. Agreed. Every matrix is validated with Regression tests cover each rule (31 in the file, all passing), and both checker modes exit 0 on the branch. |
…ients The gate froze Desktop as customer-owned by inheritance from self-managed. Self-managed is forced by redirect-URI registration; Desktop is not: every wave-1 provider supports a vendor-owned public client with a loopback redirect inside a distributed native application. Add decisions/desktop-oauth-ownership.md (accepted by the release owner on 2026-09-02), flip the desktop owner cell to langflow in all three matrices, add the entra-public-client-apps source, and thread the decision through the README, the estimate (INT-5 5 -> 5.25, total 48.75), substrate-slack, frontend-surfaces, and the connection contract. Slack needs a second PKCE-enabled app because opt-in is one-way; bot scopes stay unavailable on Desktop. Customer-owned registrations remain the override, as on self-managed. A regression test pins the Desktop cells.
Summary
Discovery gate (INT-1) for Dedicated Integrations 1.13. This PR is the reviewable home for the gate outputs. The design artifacts and release-owner decisions are complete; the gate remains open until the lfx, langflow-base, Enterprise, frontend, hosted-app, product, and platform owner sign-off rows are filled.
design/dedicated-integrations/: sourced capability matrices for Google Workspace (5 actions), Microsoft 365/Teams (8), and Slack (7); decision records; the INT-2 connection contract; frontend and operator surfaces; deferred trigger findings; and the re-issued 48.75 engineer-week estimate.decisions/desktop-oauth-ownership.md, accepted 2026-09-02): a Google Desktop client in the hosted project, the Entra desktop platform with public client flows on the hosted registration, and a second PKCE-enabled Slack app. Customer-owned registrations remain the override, as on self-managed. Only who registers the provider app varies by context; the broker, callback route, and connection contract are one design.tools/listcapture that freezes the exact tool ids and schemas required by the plan.scripts/ci/check_capability_matrices.pyvalidates the JSON Schema, evidence and decision invariants, conditional-scope references, sign-off coverage, and gate-close completion.--require-acceptednow requires nonblank Name, Date, and PR cells in both aggregate and record-level sign-off tables.No provider code, tables, routes, or UI are implemented here. Those remain INT-2 through INT-14.
Phases
drive.fileconnection-contract.mdTest plan
uv run pytest scripts/ci/test_capability_matrices.py— 36 passeduv run python scripts/ci/check_capability_matrices.py— exits 0uv run python scripts/ci/check_capability_matrices.py --require-accepted— exits 1 only for the unsigned owner rows, as intended--require-acceptedexits 0 after those signatures are recordedSummary by CodeRabbit
Documentation
Chores